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ABSTRACT ~ / 

The Teacher Education and Mathematics project, funded 
by the Women's Educational Equity Act, developed ctirr icu2,<im materials 
to increase mathematical knowledge and confidence, increase the 
perception of mathematics as a female domain, reduce nr^ath anxiety , 
and develop skills in identifying and count eracti ng s£x bias in 
mathematics instruction. To develop effective materials, a 
evaluation was to document materials' use at each stage of 
development, to assess student responses to materials, to collect 
judgment of external experts, and to collect reactions and 
suggestions of instructors who used the materials. Evalu 
explored in this paper are: reliance on obseryer descriptions 
participant ratings, effects on students and instructional 
intensive oBservat ions , effects on students of frequent requests for 
feedback and evaluative reactions, and the .extent to which 
professionals can be expected to use materials as they have been 
developed. (Author/PN) 
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. The City University of New York * position or pot>c y 

Mathematics' is a key area in the educational curriculum. -\ 
Mathematical competence is a necessity for persons who wish to function 1 
effectively in Qur increasingly technological society. Mathematics has 
been identified as, fulfilling a gatekeeping or filtering function 

because of its' importance in many careers (Sells, 1973; Burton, , 1979) . 

o • 

Sex differences in mathematical experiences both in and out of school, 

math anxiefy and sex rol6 stereotyping of mathematics as a male domain 

have frequently been cited as factors in math*avoidance for women 

(tenn ena and Sherman, 1977). Fax (197^) pointed out that. ^ 

Although anxiety about mathematics may not in every 
case be a direct result of sex-role socialization 
conflicts, it is likely that the sex-typing of * ^ 

mathematics as a male domain by parents, teachers 
and peers results in the acceptance of math anxiety 
in females as inevitable or irrelevent to their x 
development. (p. 45) * * {\ 

^Math anxiety can begin as early as the elementary school years.." 

bonady, Kogelman and Tobias (1^76) found evidence of this phenomenon in , • 

women f s reports during counseling sessions. Teachers 1 attitudes, 
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classroom activities, and sex role stereotyping in materials have ail 
. served to reinf orce^he perception of mathematics as a male domain 
(Brophy & Good, 1970; Fennerta, 1980). The need for teachers who are 
free of math anxiety and therefore able to help their students to enjoy 

i 

and to continue to study mathematics has been pointed out (Donady and 

Tobias, 1911). In order to foster students' mathematics skill develop- 

ment and positive attitudes, teachers need to perceive mathematics as 

equally important and useful fib r both girls ami boys (Skypek, 1980). 

. Teacher Education and Mathematics (TEAM) a two-year project, t 

funded by WliEA was designed to sen^i k t.ize prospective elementary school * 

teachers to the importance of mathematics for ^11 students by . developing, 

a model program and instructional ^materials. These materials were \ 

designed to reduce teacher education students' mathematics anxiety; .to, 

increase their perception mathematics as a female domain and to 

develop their skills in identif yiug ( $ex b'ias in. mathematics curriculum 

materials and teacher-pupil interaction in the classroom. (Within a< 
' « 

< teacher education program they can be used most advantageously before 
students take ^ eir course in mathematics curriculum and methods.)* This 
paper deals with the formative evaluation strategies , used during the 

I 

TEAM project » , 
TEAM Pr ogram a nd Materia ls 1 

Instructional strategies were designed by Dr. Claire Newman, 
Mathematics Educator and by the author, with the collaboration of the 
TEAM staff, ^both to foster positive .attitudes toward math and'to develop 
mathematical knowledge and skills through integrated cognitive and * 
affective learning experiences. The' inverse relationship ( between math 
an^Letjty .and. mafch knowledgT^Teporl'eS^ in the literature (Aiken', ' 1970 and 



1976; Richardson and Sulnn, 197.2 and 1973) provided the rationale for 

the program's stress on mathematical competence as well' as confidence. 

( Four mathematics modules were developed, each o.f which consists of 

ian Instructor's Text, Student Materials* Exercises, Solutions and Guide 

to Student Exercises, and Sfudent .Summary and Review. The Instructor's 

Text provides direction on what an instructor may do, suggests how 

instruction may be carried out and provides* commentary which includes 

explanations, instructional options, attitudinal interventions and 

organizational ideas. The mathematics ipodules titles are': Patterns, 

A pproximation and Estimation , Metric Measurement and Choice and Chance . 

** ' * 

Modules designed to deal with students' attitudes' consist of an * 
Instructor's Text and Student Materials. De mystifying Mathema tics deals 
with the^ individual's* own math experiences, nath myths which can serve 
as learning barriers, and math in daily life uses. Sex Role 
Stereotyping in Mathematics E ducatio n provides experience^ in, identify- 
ing *sex bias in curriculum materials and classroom practices. Women , 
Math ematics and Careers provides opportunities for students to- consider 
the role pf nathematic^ in careers and to devise experiences for chil- 
dren to alert them to mathematics ' * role in careers. Wome n as 
Ha thcmat'i c i an s introduces and describes women who have made significant 
contributions to mathematics. 

An Ins tructor's Handbook outlined the structure and purposes of the 

^ " 1 e 

instructional, material. It includes: User's Guide, Introduction (to 
the problem of mathematics attitudes), Purposes and Strategies, Learning 
Environment, Attitudinal Interventions anci Bib] iographies. 

TEAM, materials were used first at Queens College in two sections of 
a mathematics course (N = 44) during one. semester of the first year. In 



the second year, some or all of the modules were used in courses 1n five 
settings. The fall classes we're labelled TEAM II (N = 34) and the 
spring class TEAM III (N =29). 

t 

Formative Eval u at ion Plan_ ^ 

'since TEAM was essentially a, program ark materials development 
project, it was^necessary that the formative evaluation design assess 
the degfee to which the. products were, or were showing promise of, 
serving the project's goals. The formative evaluation focused on: . 
■ documenting the uses of materials at each stage of fchei* development, 
assessing students 1 responses to the materials, collecting judgments of 
materials from a panel of experts, and collecting reactions and sug- 
gestions from instructors who used the materials. 
General P rocedures t * . 

% The evaluation/revision process was viewed as ai\ interactive one, 

with evaluation staff interacting with program staff to clarify major * 
points of enphasis in student objectives and teaching activities, ffnd 
revising the^valuhfion data collection instruments to reflect these 
goals. ,Blenor Rubin Denker, who served as TEAM 1 s Evaluation Associate 
provided feedback quickly to the staff on all data collected so that~Ttr 
could be considered in planning sulj^equent instructional sessions. 
I nstru ments . ^ 

Data gathering procedures were devised to serve specific project; 
needs. Tn the fijst year of the project, during *the initial materials 
development, a Rating Scale for Modul es was devised which was used by 
consultants in their appraisal of the draft modules* This 13 item scale 
included ratings on the materials 1 usefulness, appropriateness for, • 
teacher education students, clarity,, comprehensiveness, length, etc. 
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Ratings were n»ide on a 5 -point scale, ranging from "highly satisfactory" 

to "needs major revision. 1 * Thes/ ratings provided the basis for re- 

visions and a second draft of the sample modules. 

A Background Questionnaire which dealt with student's high school, 

and college mathematics course history, perceptions of family attitudes 

toward* mathematics, ratings of mathematics achievement, attributions of 

selected mathematics course grade^ and purreht feelings about studying 

1 mathematics was devised and piloted during the first Semester of the • 

project, and used in the second semester. Student responses ' to t this 

questionnaire provided a profile' of the t teacher education students and 

finally of TEAM, students which the instructors used in developing plans* 

for specific learning experiences and class sessions; 

During the seconti half of the first year, when the first group of 

/ students were enrolled in the TEAM course, an Initi al Struct ured, - 

Interview was devised which containeda wide range of items of concern 

-to the project, i.e. students 1 current feelings about mathematics, their 

mathematics learning history and details of any difficulties they had 

encountered and who had helped them with their difficulty, their 4 pe^cep- 
■ * * » 

tion of their family's interest in mathematics study, their Anticipation 

of their feelings about studying mathematics currently and what they 

expected and wanted to get from the TEAMprogram. These interview data 

helped the staff to understand students' .needs and starting points in 

the program. Until these data' were available, specific needs of the 

TEAM students were unclear so/' projections and plans could ^pe made only 

/ ** 
on Che basis of the staff's k'perlences with former students. 

,&v • h 

An EnA_oI_^.^^---^fil^lHSi-— ' using 3 cricical incident a P~ ' 1 
"proach, asked what the best/ and worst things about the TEAM course were 



\ * / ^ . - • 

and aske3 students to identify factors to whicH>tney woul'd attribute a 

r ' •/ , \ 

good arid a pooY math grade.' An End of Sem ester Interview dealt frith 

.__ A <r- . . 

each student's description* of her currenf math attitudes, current rating 

of math anxiety, changes (if an^) in attitudes toward math, reactions to 

mathematical situations and s any changes noted in the way a student 

noticed math in daily life, reanalysis bf personal math \history to share 

any new* understandings* developed about influences on attitudes, per- 

ceptjions Qf tfeachers 1 influence op girls 1 mathematical* self-confidence, 
I 

feelings about the way females f are portrayed in math texts, anticipated 

feelingp about teaching math, feelings about being in* an experimental 

course, and, recommendations for the TEAM course. While these data 

i 

fulfilled the function of assessing the effects of the program and 

* «• ** 

materials on the first group of students* th'ey served to provide /£eed- 
J/ack to the staff abotft which areas were satisfactory and which needed 
modification, ** 1 ' x 

A Katjng Scale for Cljiss Sessi ons was developed and revised to 

, . ^ • > 

provide ratings of, math content (i.e. interesting, useful, new to you, 
understandable, frightening, hard) and ratings of the general* atmosphere 
of the class (i.e. helpful, reassuring and enjoyable). Students 1 
comments were als"o requested in the form. This scale was used at the 
end qf the orientation session and After classes 3, 6, and 12. 

During the second year or field trial*. stage, two questionnaires 
were developed in parallel form and used as 'a pre- and post-semester 
instrument to replace the interviews and questionnaires used tp the * 
first year. Since the materials were now to be used by teams other than 
the developers in settings in which the cost benefits bf activities, were 



important for the new users, it was important to provide procedures ttyat 
wouW be useful and "not excessively costly in faculty time and effort. ' 

The rating scale for classes which had been used in the first year 
was maintained as the instrument to gather students 1 responses to their 
class experience. * . 

Another data source in year two were Logs i>n which students were 
asked to write a reaction each week A tfhich< emphasized their feelings 
about math. They were free to write about anything they wanted to * 
discuss or needed to, express. Student Lcrgs were also kept during the s 
TEAM ITT classes. Instructors stressed the nonevaluatiye nature of 
these Logs, that is they were not to be considered in assigning a . 
student 1 s grade for the course. " <■ 
^S^-^^Inst rue tors 1 reactions to the materials were 'sought through post 
session interviews by the person who was the assigned observer or by a 
staTf member. These interviews included assessments of the materials 
used as well as suggestions" for modifications. 

Observations . * - . 

\ 

"During the first year dr, development £tage, observational data were 

collected in class sessions by at least two observers. In general, we 

* \ i 

avoided tj\o problem of inter-observer reliability by assigning one 

\. * * • 

observer \to a task. In some sessions there were observers* each with a 

specific tasK All of these .data were being gathered for the. purpose' of 

"x 

\ 

, revising the materials be.fore the field trials. - 1 - 

Participation data wete tabulated, i/e. numfcie* of student speakers 
tallied during half-hour intervals, throughout the classes. These data 
were 'gathered because the goal of the project Was to create conditions 

\ 

. t 
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in which student's would actively participate. Verbal statements were ' 

used as an indicator of active^articipation* 4f 

■» 

Indications of anxiety were recorded by one observer who noted 

r 

behavior such as 'giggling, restlessness, whispering, glancing around 

etc, and rated the degree of anxiety on a 4-point scale. ' Recognising 

, - ' \ ■ . . ' 

that these may also be signs of inattention due to factors o.thor than 



anxiety, we decided t£ continue to gather these indicators andj/to note 

the content or topics periodically to see if these behaviors varied in 

« 

relation to content, 

* * 

A Log of each class session noted changes from the proposed math 

instructional sequence, times , ♦students 1 quest ions^or areas of con- , J 

fusion, additional explanations provided, additional illustrations used, 
etc. Statements' specifically directed to students' attitudes; anc) the ' \, 

behavior or incidents t-hat lead *to t-hem were noted along with the nature 

j 

of the discussion that followed. During the second ye^r of the project, 

* • v - / 

this observation plan was reduced and simplified,. One person kept a \og/f 

' . 1- . . 

of the flow of each session and noted'the *tctpic , activity and class |# ' * / 

structure and anxiety at 10 or 20 minute intervals*, * 
Findings and Observ ations j 

While the* Rating Scale for Modules yielded some useful data in that 

4 I . , 

V 4 * 

the degree to which a module wa$ satisfactory on the specified criteria \ 
was indicated, the most useful information came in the comments, and *,A 
suggestions f-or modification. Discussions with consultants provided the \ t 

\ j - / . 

richest source of information on module modification. The Bating Scale* 
* 

and its criteria were useful, perhaps, in directing consultants 1 atten- 
^J»on to features considered important by the -deyelopers. ' ^ 
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Interview and Questionnaire fdata proved useful in^ describing the 
nature ©f the student group. In general, TEAM students were college 
juniors who had not taften mathematics courses since high school , who 
reported negative feelings and experiences about mathematics and who 
wanted to increase their achievement and confidence in -working -with 
mathematics. The students who were recruited for the classes were the 
-ones the project had intended to serve. The variation found in age and 
ethnicity in the TEAM student group was considered desirable, since it* 
had been the project's intent to develop materials that would be useful 
in diverse groups. * 

V • • - 

Ih general, the End of Semester ^Questionnaire and Interview .data 

provided a picture of gair^s toward the project goaig^e*g. 61% of the 

"group rated themselves as less math anxious,. 78% Indicated, that their 

general attitudes toward math had changed and were highly positive (30%) 

or somewhat 'positive (45%), 93-% indicated awareness of sex stereotyping 

* 

n * 

in curriculum materials, etc. Student's responses alerted the staff to 

j • t 

the ^var^ous^ ways in which students were conceptualizing* frnd annmarizing 

their .experiences. Some people (39%) dealt with confidence ii/ their 

ability to learn math, while others (21%) described the classroom 

atmosphere as supportive, and still others (16%) focused on their nath 

achievement. Problem areas were ' identified through the variations in 

students' perceptions of the difficulty of-*the math content ifrom one 

module to another, and in the appropriateness of the math content' fo# 

some individuals. These reactions were considered in preparing the * 

revisions of the modules 'for the Second year. » . 

The observation data dealing with the content and flow of the* 

sessions proved, particularly, useful in module revision. Using these 
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data, some module sections were varied to provide more illustVative 

m 

materials* while others were reduced; more exercise materials that 
involved* simpler applications of concepts w^ere included , etc. 

Participation data showed -that an average o'f 75% of the students 
spoke at leasts once during each* class session. 

The observations of anxiexy indicators turned-up a particularly 
intriguing set of behaviors. Preening was noted during #a highly anxious 
time when preparation for a quiz was being discussed. At that time, 
students were observed taking out hair brush.es, nail files, lipstick 
etc. and grooming themselves during the discussiop. It. would almost 
seem as if some of the women in the cVafrg were seeking reassurance that 
they would be acceptable in the face of the th-rcateiVed loss df \ ], 
self^est'c&n posed by the quiz/ 

Bottj more frequent and longer students' ratings of the class 

sessions Jiad been planned than actually occurred, ^nitially, eacti 

session was to have been rated, with a longer rating scale, -but the - 

staff §#t?tled for having each module rated .with the revised, simplified 

scale described above. Students were willing to provide the ratings but 

>they communicated in subtle«.ways that they were more interested in doing 

the class activities than in rating them. And too, the pressure on the 

evaluation staff; to handle the quantity of' data was a consideration in 

* 

reducing the ^number of rating's. Ratings showed generally that students 
. found the , sessions satisfactory and found the general a tmb sphere s,up- 
, portive. The comments which students offered provided useful data Tor 

sp^cifit modifications that coufol be considered by the*staff. Knowing 



the groups 1 averages in ratings provided a framework within which such 

* • • * 

individual suggestions could be consider^!. \ 
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. During the second year, Logs provicfed a steady flow of information 
f on student^' facings and reactions. ^This device proved ' especially 
useful as feedback for the class its.elf. * For example, the tendency 
wljich the Logs revealed for students to explain their successes on 
quizzes as a function 'of the quiz's being'easy rather than of their 
mastery of the materials was described to the class* An interesting * 
discussion of the factors to which the students attributed their sue- 
cesses and failures' followed. The Logs also gave the staff prompt 
feedback on the students' reactions to classes, assignments, mathe-' 
matical situations and incidents, etc. Areas that needed sta£f atten- 
tion as well as the degree to which students were 'facing and dealing 
with their anxieties was evident in their statements. % . 

Issues and , Suggest tons 

The complexity of"*telling the story of a case, of the descriptive 
narrative, has been detailed by Smith (1978). Trying to catch the 

f r 

essence of what was transpiring in the classes was our goal. One of the 
advantages of the design we used was the inclusion of groups over three 
semesters. This moved us beyond some of the limitations of the 
"one-shot case study" ancl it, treated problems in our knowing whether the 
changes in groups' responsiveness were a function of the improvements 
made, in the materials and programs or o£ the .di f f erences b-etween the 
groups. 

* • i% >; K « : 

Among Xhe issues that became apparent during the use of the forma- 
ti.ve evaluation strangles described above was the heavy reliance of our 



. t ° 

• s tfes 



data on observer s inscriptions and participant and consultant ratings. 

r * 

These ratings were limited by the factors or criteria that had been 
identified for fo^us and £he nature of the scales constructed for use;' 



Whil ( e the criteria selected and items constructed were related logically 



to the projec t t f § objectives ahd included tactors Qsuarlly considered ' 

evaluating x:urric:ulum materials, some, x uncertainty remaips about the , ' 

. V ' ' " 

validity of the ratings. I-t should be noted, ^owever , ' that while we- 

* * *■ • % 

were in 1 the midst of the .process, we tended ,to look upon our ratings and 

, . . jfc • 

observational: <i3a'ta as "real 11 indicators of the phenomena under 

y 

.consideration. "Projeqt preSs," described by Smith (1977)" was clearly a* 

factor in making early decisions^ about the foci of the various r 

observations and rating scales. Our limited resources^ .gave rise to ? 

caution about committing heavily to an ethnographic model. We 

compromised and selected variables seetaed promising. « 

i 

' The presence of numbers of observers in the class sessions presents 
particular problems. Tension^ were engendered in some of the partici- 
panCs despite detailed explanations that the purposes of the observation 

'.J 

were to assess the materials and the instructor's procedures. While 
many of the students were able to forget, or , ignore the^observer 1 ^ 
presence, others were not able to do so; they felt that,\they were in 
some way being studied ot watched. It seems reasonable to conclude that 
a* class is changed in some ways by the presence of an observation team. & 
This issue has particular salience since the ctfrrent increase K+r 
ethnographic classroom studies has brought and will continue -to bring 
observers into classes. It may be that the psychological' nature of some 
of the classes' content lead to students 1 increased reactivity to the : 
observers-' presence, and that the brief, intense nature of the project 
didn't permit for a kind of adaptation to the observers as "just part of 
the scene 11 for some people. 



Participant attrition in TF.AM was minimal! .One factor -which may 
have contributed to this was the students' having volunteered to partic- 
ipate. Having the program carried out* in a class for which students 
were earning credits probably contributed to their commitment and served 
to maintain the group intact. Attrition is a factor of considerable 
.consequence in experimental programs which have j^ ss structure. 
; The effects on students of frequent requests for feedback and 

* 

evaluative reactions are unknown. It seems' plausible that these re- 
it • 

* 

quests increased students' self-consciousness. .In some sense they 

*i * 
became both the subjects and objects of study. * While some students 

responded very well to this and seemed to grow in* their ability .to 

identify and express their feelings, crthers did not do so. For example, 

on Log entries, some students took a "Dear Diary" approach and were able 

to sense and describe what was happening to them. Others, in contrast, 

described the realities or external features of situations rather than 

their feelings* or reactions. The Logs ap/ear to yield useful data, as 

well as to provide an interesting growth experience for some people* 

Some students seemed amenable to providing descriptions and ratings 

while others were much more interested in spending their time in 

activities. ^ , 

The extent to which professionals can be expected to implement a 

r 

program and use-ignate rials developed by others #in the recommended manner 
$eems to depend upon many factors. The assumption underlying the 
establishment of field trials for materials is that they will be used in 
the ways in which the developers intended. It seems more reasonable to 
assume that strict adherence to a prescribed set of materials and 
procedures is npt necessarily demanded — rather that instructors can 
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interact with the materials and use them effectively wit.fe a, heightened 
sensitivity and awareness of students 1 needs. During the *TEAM trials, 
instructors operated with the spirit along with many of the specifics of 
the program, thajt»is they were modeling an approach which we want the 
students to take when they become teachers. It is also of interest to 
-note that the levels of u£e of an innovation (Hall et al., 1975) place 
ief inements "the^user varies the use o£, the innovation to incr^se the 
impact oft clients within the immediate sphere -of influence" at a higher 
point than r outine use , . . few if any changes are being made in 
ongoing, use 11 (p. 54). This emphasiEes the value of materials being 
used to serve e f f ect ively ' tih£ nee d s of pa r t icutar st uden t s . 

In .summary, the TFAM formative evaluation d&sigri had as its pur- 
poses the*documentation of^-ffST»— at each stage of materials 1 development, 

L • 

the assessment of stucyent responses to the materials, the collection of 
judgmental responses from external experts and the colle6t$^S-of re- 
actions and suggestions from instructors who used the materials. Issues 
raised by these strategies were identified. 
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